This invention is related to a resolution conversion system for converting digital graphical information of still pictures and moving pictures including characters, line drawings, natural pictures and the like into another graphical information having a different number of pixels so as to display converted pictures on a display device. More specifically, the invention is related to a resolution conversion system and the method thereof which allows input image information of a particular number of pixels to be displayed on the display device or a screen in a window environment having a different resolution when displaying the multimedia moving pictures.
In the conversion of digital images, when the number of pixels of an original image is reduced, another image of a reduced scale from the original image can be obtained on the same display device. Accordingly, it is useful when displaying a reduced image in a window or the like to reduce the number of the pixels of the original image. In addition, it is possible to display images on a screen of a display device capable of lower resolution than that of the original image.
In order to reduce the number of pixels of the original image, the simplest way is to thin some pixels out. Alternatively, there is another way to reduce the number of the pixels of the original image by simply calculating the mean of luminance values of 3xc3x973=9 pixels (e.g. if each pixel has 8 bits of luminance information, the luminance value falls within a 256 level gray scale) and by assigning it to a luminance of one pixel.
Furthermore, in Japanese Patent Application Laid-Open No. Hei 6-124189, there is disclosed a method for reducing the number of the pixels of the original image by taking averages of the image data of the original image in a horizontal direction and a vertical direction. In this method, lines are drawn using a line drawing algorithm known as a conventional art, and the horizontal line of the original image is shown as a set of a plural dots (y dots) of a horizontal component. The length of each horizontal line may be considered to be indicative of the number of input pixel data sets corresponding to one pixel data set after scaling-down to be stored in VRAM. Although the lengths of the horizontal lines may be different from each other or they may be identical to each other, the horizontal scaling-down is accomplished by taking the average of the pixel data for n dots. The vertical scaling-down can also be similarly accomplished.
On the contrary, if the number of pixels of the original image is increased, an enlarged image of the original image can be obtained on the same display device. Accordingly, it is useful for displaying an enlarged image in a window or the like to increase the number of the pixels of the original image. In addition, it is possible to display images on a screen of a display device capable of higher resolution than that of the original image.
In order to increase the number of the pixels of the original image, the simplest way is to surround a pixel in the original image, in horizontal and vertical directions, with pixels of the same gray scale level.
Alternatively, instead of simply placing the pixels having the same gray scale level, there is another way of increasing the number of pixels of the original image by calculating interpolation pixels whose gray scale level is positioned on a line extending between adjacent pixels of the original image between which the pixels are interpolated so as to use the calculated interpolation pixels.
Furthermore, a method for arbitrarily converting the resolution of the original image is shown in FIG. 1. In FIG. 1, 2001 designates original image data, 2002 designates a process for multiplying the original data by L1 in a vertical direction and by M1 in a horizontal direction, and 2003 designates the image data subjected to the L1xc3x97M1 multiplication processing. Reference numeral 2004 designates a process for multiplying the enlarged image data by 1/L2 in the vertical direction and by 1/M2 in the horizontal direction, and 2005 designates the image data subjected to the (1/L2)xc3x97(1/M2) multiplication processing, wherein L1, L2, M1 and M2 are positive integers. In the multiplication processes of L1xc3x97M1 and (1/L2)xc3x97(1/M2), if L1 greater than L2 and M1 greater than M2, then the image is enlarged, whereas if L1 less than L2 and M1 less than M2, then the image is reduced. In this way, the size of the image are arbitrarily modified in accordance with the combination of the enlarging and reducing processes.
If the same pixels as one pixel of the original image are placed in the horizontal and vertical directions so as to increase the number of pixels in accordance with the above mentioned method, the obtained image may include very noticeable aliasing, and therefore the quality of the obtained image may be degraded.
Instead of simply placing the same pixels, if the number of pixels are increased using the interpolation pixels whose gray scale levels are positioned on the line which links gray scale levels of the adjacent pixels of the original image, a very noticeable aliasing mosaic image is prevented from occurring. However, since high frequency components in a portion of the original image where a boundary between light and shade is clearly recognized may be reduced so as to smooth that portion, the obtained image becomes faded or blurred.
Furthermore, according to the method for arbitrarily converting the size of the image using the combination of the enlarging and reducing processes, since the reducing processing is executed after the completion of the enlarging processing, the processing is time consuming. In addition, according to this method, since the image is enlarged prior to the reducing processing, it is necessary to provide a memory for storing data of the enlarged image, which makes the hardware large.
The first object of the present invention is to prevent an image having noticeable mosaic aliasing so as to realize a high quality image, in resolution conversion increasing the number of pixels of an original image.
In addition, the second object of the present invention is to store a high frequency component of a contour portion of an original image where a boundary between light and shade is clearly recognized, so as to realize a high quality image, in resolution conversion increasing the number of pixels of the original image.
Furthermore, the third object of the present invention is to realize a high speed resolution conversion processing which can be used for a digital moving picture processing, in a resolution conversion increasing the number of pixels of an original image.
In addition, the fourth object of the present invention is to realize arbitrary resolution conversion processing in a minimum size of hardware, in resolution conversion increasing the number of pixels of an original image.
The above and the other objects of the present invention are accomplished by a resolution conversion apparatus for converting an original digital image into a digital image having a different number of pixels in accordance with an instructed conversion magnification factor comprising a determination circuit for determining, for each block including a predetermined number of pixels of the original digital image, the number of interpolation pixels and positions where the interpolation pixels are interpolated in the block of the original digital image, and a converted image generation circuit for generating pixel data for the interpolation pixels to be interpolated in each of the blocks in accordance with the number of interpolation pixels and the positions, so as to obtain a converted digital image based upon pixel data for the pixels in the original digital image and of the generated pixels to be interpolated, wherein the converted image generation circuit is arranged such that when a difference between two pixel values in the original digital image adjacent to each other at each of the positions where a pixel will be interpolated is equal to, or less than, a predetermined threshold, the values of the pixels to be interpolated are generated on the basis of the pixel values of the two pixels in the original image and a predetermined first interpolation equation, while when the difference is greater than the threshold, the values of the pixels to be interpolated are generated on the basis of the pixel data values of the two pixels in the original image and a predetermined second equation.
According to the present invention, in an area in an image including more high frequency components, where light and shade of the image is clearly visible, attention is paid to the fact that an absolute value d of a difference between values (for example, gray scale levels) of two pixel data sets is greater than a predetermined value. In a case where the absolute value d being the difference between image data of two pixels is less than the predetermined value, as in a case of an area in an image including more low frequency components, where light and shade of the image is not particularly discernible, a set of values for interpolation pixels are generated, which are positioned on the first interpolation equation (for example, a linear equation passing through values of pixel data for two pixels) which does not maintain, that is, which eliminates, the high frequency components of the image.
On the other hand, in an area in an image including more high frequency components, where light and shade of the image is clearly visible, when an absolute value d of the difference between two pixel values is greater than a predetermined value, a set of values for interpolation pixels are generated, which are positioned on the second interpolation equation (for example, a curve equation of a spline function passing through values of pixel data for two pixels) which maintains the high frequency components of the image, by comparing with the first interpolation equation.
According to this process, generation of an image having noticeable mosaic aliasing is prevented so that a high quality image can be realized. In addition, even in a case of converting a natural image whose contours are blurred, a high quality image can be realized, and in a case of converting letters or line drawings whose contours are clearly visible, a high quality image which has a clear boundary of light and shade can be obtained.
In a preferred aspect of the present invention, the converted image generation circuit includes a difference calculation circuit for obtaining a difference between values of two pixels, a pixel data calculation circuit for calculating pixel data values for pixels to be interpolated based on the first and second equations, and a selection circuit for selecting the pixel values calculated by the pixel data calculation circuit based on the first equation when the difference obtained by the difference calculation means is equal to, or less than, a predetermined threshold, and for selecting the pixel values calculated by the pixel data calculation circuit based on the second equation when the difference is greater than the threshold.
In another preferred aspect of the present invention, the converted image generation circuit includes a difference calculation circuit for obtaining a difference between values of two pixels, a selection circuit for generating a first selection signal indicating that the pixel values are to be generated based on the first equation when the difference obtained by the difference calculation circuit is equal to, or less than, a predetermined threshold, and for generating a second selection signal indicating that the pixel values are to be generated based on the second equation when the difference obtained by the difference calculation means is greater than the threshold, and a pixel data calculation circuit for calculating pixel values for pixels to be interpolated based on one of the first and second equations in accordance with one of the first and second selection signals.
In a further preferred aspect of the present invention, the determination circuit determines the conversion magnification factor as conversion magnification factors of a horizontal direction and a vertical direction, and the generation circuit includes a horizontal generation circuit for generating the pixel values of pixels to be interpolated in the horizontal direction and a vertical generation circuit for generating the pixel values of pixels to be interpolated in the vertical direction.
According to this aspect of the invention, since it is possible to execute resolution conversions in the horizontal and vertical directions, respectively, it allows the hardware necessary for resolution conversion processing to be reduced in size.
In another aspect of the present invention, the resolution conversion apparatus for converting an original digital image into a digital image having a different number of pixels in accordance with an instructed conversion magnification factor comprises a determination circuit for determining, for each block including a predetermined number of pixels of the original digital image, the number of interpolation pixels and positions where the interpolation pixels are interpolated in the block of the original digital image; and a generation circuit for generating, for the block of the original digital image, pixel values for the interpolation pixels to be interpolated in the block in accordance with the number of interpolation pixels and the positions where the pixels are interpolated, which have been determined by the determination circuit, so as to obtain the converted digital image, using pixel data values of the pixels in the original digital image and of the generated pixels to be interpolated as a basis.
Furthermore, in another further aspect of the present invention, an image information processing system comprises an image input circuit for receiving image data, a storage circuit for storing pixel data values inputted from the image input circuit, a magnification factor receiving circuit for receiving a conversion magnification factor used for converting the number of pixels constituting the pixel data values into another different number of pixels, an image processing circuit for reading out the image data stored in the storage circuit, and for converting the read out image data in accordance with the conversion magnification factor provided by the magnification factor receiving circuit, and an output circuit for outputting image data values converted by the image processing circuit, wherein the image processing circuit includes a determination circuit for determining, for each block including a predetermined number of pixels of the original digital image, the number of interpolation pixels and positions where the interpolation pixels are interpolated in the block of the original digital image; and a converted image generation circuit for generating pixel data values for the interpolation pixels to be interpolated in each of the block in accordance with the number of interpolation pixels and the positions where the pixels are interpolated so as to obtain the converted digital image, using pixel data values of the pixels in the original digital image and of the generated pixels to be interpolated as a basis, wherein the converted image generation circuit is arranged such that when a difference between values of two pixels of the original digital image adjacent to each other at each of the position where the pixels are interpolated is equal to, or less than, a predetermined threshold, the pixel data values of the pixels to be interpolated are generated using the values of the two pixels in the original image and a predetermined first interpolation equation as a basis, while when the difference is greater than the threshold, the pixel data values of the pixels to be interpolated are generated using the values of the two pixels in the original image and a second interpolation equation which is different from the first interpolation equation as a basis.
Furthermore, the objects of the present invention are accomplished by a resolution conversion apparatus for converting an original digital image into a digital image having a different number of pixels in accordance with an instructed conversion magnification factor comprising a determination circuit for determining, for each block including a predetermined number of pixels of the original image, positions where interpolation pixels are to be interpolated, in the block of the original digital image, in accordance with the conversion magnification factor, and a converted image generation circuit for generating pixel data value for the interpolation pixels to be interpolated in the determined positions in the block in accordance with a predetermined interpolation equation whose coefficient is determined with respect to positions of each pixel included in the block and values of the pixels for every block of the original digital image so as to obtain the converted digital image based upon pixel data values of the pixels in the original digital image and of the generated pixels to be interpolated.
According to the present invention, the determination circuit determines positions, in the block, of interpolation pixels to be interpolated in the block of the original digital image, for every block having a predetermined number of pixels of the original digital image. In this case, it is possible, for example, to store positions determined corresponding to respective conversion magnification factors in tables, and determine positions according to the conversion magnification factor by making reference to the tables. In addition, a pixel block unit may be a block of 8xc3x978 pixels.
Furthermore, the converted image generation circuit generates pixel data for the interpolation pixels to be interpolated in the positions in the block, which were determined by the determination circuit, in accordance with a predetermined interpolation equation whose coefficient is determined with respect to positions of each pixel included in the block and values of the pixel data sets for every block of the original digital image so as to obtain the converted digital image in accordance with values of the pixels in the original digital image and of the generated pixels to be interpolated. The interpolation equation may be a spline function or a Bezier function. According to this, values of the interpolation pixels can be optimized.
In a preferred aspect of the present invention, the determination circuit determines at least one of the interpolation positions to be provided between a block where interpolation pixels are to be interpolated and a block adjacent thereto, and the generation circuit determines a coefficient of the interpolation equation based on positions and values of pixels adjacent to the block where the interpolation pixels are to be interpolated, so as to generate values of the pixels to be interpolated.
According to the aspect, it is possible to provide interpolation positions between a block where the pixels are interpolated and a block adjacent thereto and to determine a coefficient of an interpolation equation by further using positions and values of pixels in the adjacent block, in addition to positions and values of pixels in the block where the pixels are to be interpolated. Accordingly, the interpolation pixels can be generated at a portion where one block is adjacent to another, that is, outside of the blocks, so that it is possible to prevent the values of pixels from being discontinuous even at a position where blocks are adjacent to each other.